1. Field of the Invention
The present invention relates to object based attention tracking in a virtual environment.
2. Background Information
Eye-tracking within this application will reference the point of gaze of a subject, rather than associating the motion of the eye relative to the head that is also referenced as eye tracking (although these two eye tracking items are clearly related).
Eye-tracking has been used in “usability testing” for many years, possibly the first practical applications with cockpit design testing in 1950. Such early work has been valuable in establishing some assumptions about the relationships between eye-movement data and the cognitive activity of a user. Frequency of fixations was assumed to be related to the importance of the control, while the duration of fixations was related with ease of interpreting information.
Alfred L. Yarbus (sometimes spelled Iarbus) was a Russian psychologist who studied eye movements in the 1950s and 1960s and noted “records of eye movements show that the observer's attention is usually held only by certain elements of the picture. ( . . . ) Eye movements reflect the human thought processes; so the observer's thought may be followed to some extent from records of eye movements (the thought accompanying the examination of the particular object). It is easy to determine from these records which elements attract the observer's eye (and, consequently, his thought), in what order, and how often.” See A. L. Yarbus, Eye Movements and Vision. New York: Plenum Press, 1967. (Translated from Russian by Basil Haigh. Original Russian edition published in Moscow in 1965).
Eye tracking methodologies have been exploited successfully by the print advertising media for several decades. Understanding where a viewer's attention will be directed on static image has been utilized to maximize the effectiveness of the static image, e.g. printed, advertisement. This is merely one use of attention tracking developments which is on the commercial side. Other research applications have been pursued in this field.
The advent of effective computer monitor based eye tracking systems, such as the Tobii T/X™ series of eye trackers from Tobii Technology AB, have significantly improved the abilities of attention tracking review of static images on a computer. Such eye tracking testing provides unique methods to assess the impact of advertisements and web pages. It is believed that where people look accurately reflects their attention, thinking and what information they are processing. Automated eye tracking provides insights that cannot be obtained directly with other testing methods.
It is asserted that by effectively testing a proposed design (such as a print advertisement, product design, or webpage) before launch with an automated eye tracking system, the users are able to greatly improve its impact and avoid large spending on suboptimal design. In contrast to many design testing systems today, such as focus group studies, automatic eye tracking provides objective results. It is asserted that by observing people's eye gaze, a true measure of responses and reactions is obtained without the filtering of the respondent's logical mind or the influence and interpretation of a test leader. It is asserted that such automatic eye tracking provides both qualitative and quantitative results that allow users to gain clear insight and effectively communicate design implications: for example a user can (i) Observe how a subject's eyes wander across a design, in real time or after testing, to obtain a deep and direct understanding of reactions and cognitive thought processes; (ii) Show visualizations like gaze plots and hot spots to effectively illustrate how individuals or groups of people look at a user's design and where to place valuable content, and (iii) quantifiably identify and back up conclusions about what people see and for how long.
As a representative examples of the objective results of typical automatic eye tracking studies, FIG. 1 illustrates a prospective gaze plot 4 of a print advertisement, or static image 2, obtained from and automatic eye tracking study and FIG. 2 is a graph 6 associated with this print advertisement review. FIG. 3 is a hotspot plot 8, also called heat plot, of a collection of subject studies of a prospective webpage or static image 2. These representative illustrations may be found at www.tobii.com along with further descriptions of the abilities of automatic eye tracking for static images 2. Within the meaning of this application eye tracking data, gaze data, hotspot, gaze plots and the like will be collectively referred to as attention tracking or attention data.
Similar automatic eye tracking systems have been applied to video streams as well. The systems work very well at tracking what portions of the screen or monitor the subjects were looking at any given time throughout the session. However, unlike the still images, this data cannot be easily associated with particular three dimensional objects or areas of interest (AOI) within the video stream. In most videos, by their nature, the three dimensional objects move around relative to one another on the monitor.
Some attempts have been made to co-ordinate the eye tracking data of a video stream with the particular three dimensional objects of the video through a laborious hand mapping procedure. In such a mapping procedure the eye tracked data is reviewed in a frame by frame manner and the gaze data is then assigned on a frame by frame basis to specific objects within the video. This process is extremely time consuming so as to make large data samples impractical, further it introduces the subjective issues of the person assigning the gaze data to an object. For example, two different researchers analyzing the same frame may assign the same data differently to two separate objects, e.g. one assigns the data to the face of a person in the foreground and the other assigns the gaze to an automobile in the background of that frame.
There is a need for efficiently, effectively and objectively assigning eye tracking data to moving objects in viewed stimuli. The ability to provide such a tool to researchers and the like will open the door to a greater number of applications than used with still images, as we live in a moving three dimensional world.